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Abstract 

We show that the Parikh image of the language of an NFA 
with n states over an alphabet of size k can be described as 
a finite union of linear sets with at most k generators and 
total size 2°( k lo s n \ i.e., polynomial for all fixed k > 1. 
Previously, it was not known whether the number of gen- 
erators could be made independent of n, and best upper 
bounds on the total size were exponential in n. Further- 
more, we give an algorithm for performing such a transla- 
tion in time 2°^ k io s( kn )). Q ur proof exploits a previously 
unknown connection to the theory of convex sets, and estab- 
lishes a normal form theorem for semilinear sets, which is 
of independent interests. To complement these results, we 
show that our upper bounds are tight and that the results 
cannot be extended to context-free languages. We give four 
applications: (1) a new polynomial fragment of integer pro- 
gramming, (2) precise complexity of membership for Parikh 
images of NFAs, (3) an answer to an open question about 
polynomial PAC-learnability of semilinear sets, and (4) an 
optimal algorithm for LTL model checking over discrete- 
timed reversal-bounded counter systems. 



1 Introduction 

A semilinear set is any subset of N fc that can be de- 
scribed as a finite union of linear sets over N k of the form 
{v + S™ x^Vi : ai, . . . , a m G N} for some offset v G N fc 
and generators Vi, . . . , v m G N fc . The well-known Parikh's 
Theorem states that semilinear sets are effectively equiva- 
lent with the sets of letter-counts (a.k.a. Parikh images) of 
regular languages and those of context-free languages |23|. 
This theorem is well-known to be a fundamental result in 
automata theory with a plethora of applications. These in- 
clude verification [T3] [TSJ [30) , automata and logics over 
unranked trees with counting [26|, equational horn clauses 
ll32l . and word-automata theory itself J5] [14] [20), among 
many others. Most practically-motivated applications (e.g. 
H Ol El [ll E0l |2§1 EQl however, require more than 
the effective equivalence of such representations. The is- 
sues of succinctness and complexity of translations among 



different representations are also equally important. These 
issues are the main subject of this paper, where nondeter- 
ministic finite automata (NFA) and context-free grammars 
(CFG) are adopted as representations of regular languages 
and context-free languages (respectively). [As we shall see, 
our results also hold for other standard representations of 
regular and context-free languages.] 

There is a trivial polynomial-time translation from a 
given semilinear set S (where numbers are given in unary) 
to an NFA or a CFG whose Parikh image represents S. On 
the other hand, the reverse (more important) direction is not 
yet fully understood. All known translations from NFA and 
CFG to semilinear sets (e.g. see Ho] [HI [23] [26) and the 
references therein) yield at least exponentially many linear 
sets. It was not clear whether (and perhaps, to what extent) 
such an exponential blow-up can be avoided. 

Some partial answers are available. Chrobak-Martinez 
Theoremj (6] [20) shows that, given an NFA with n states 
over a unary alphabet (i.e. with one letter), one could com- 
pute in poly-time a union of 0(n 2 ) many arithmetic pro- 
gressions {a + tb : t G N} such that a = 0(n?) and 
b = 0{n). This theorem has been applied to derive opti- 
mal algorithms for other problems in automata theory (e.g. 
see [ 14, 20|) and, recently, in the verification of one-counter 
systems (e.g. see HI)). Note that arithmetic progressions 
are simply linear sets with exactly one generator. Chrobak- 
Martinez Theorem is in stark constrast with the known gen- 
eral translations from NFA (e.g. JTOj [H [23] [26] [32) and the 
references therein), which produce a union of exponentially 
many linear sets with unbounded number of generators even 
over unary alphabet. On the other hand, it was shown in 
ll24l that a CFG G over unary alphabet {a} could be used 
to encode the language {a 2 }, where n is roughly the size 
of G. Therefore, at least without allowing binary represen- 
tation in the output, the size of the semilinear sets could be 
exponential in the size of G. 

Chrobak-Martinez Theorem suggests one obvious gen- 
eralization: given an NFA with n states over an alphabet of 
any fixed size k > 1, one could construct in poly-time a 
union of polynomially many linear sets with at most k gen- 



1 Unfortunately, their proofs contain a subtle error, which were only 
recently fixed in (; 29l 
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erators whose offsets and generators contain only numbers 
of size (in unary) polynomial in n. Such a generalization 
would be interesting for two reasons. First, the size of the 
alphabet is often much smaller than the number of states 
in the NFA. Second, Lenstra [ 19] has given a poly-time al- 
gorithm for integer programming over any fixed number of 
variables. As we shall see, this would yield better complex- 
ity in applications requiring the use of Parikh's Theorem. 

One immediate hurdle in proving this general version 
of Chrobak-Martinez Theorem is that it was not known 
whether for every fixed k > 1 there exists a poly-time al- 
gorithm, which given a semilinear set in N fe (in unary), out- 
puts an equivalent finite union of linear sets with at most k 
generators and total size polynomial in the given semilinear 
set. For k = 1, this is simply a corollary of solutions to the 
Frobenius problem 11251 . For k = 2, this has been proved by 
Abe 1 1 ]. However, his proof makes use of geometric facts in 
R 2 that are not available in higher dimensions. In general, 
it was not known whether the weaker statement requiring 
only existence actually hold for all fixed k > 1. 

Contributions We prove the general version of Chrobak- 
Martinez Theorem: given an NFA with n states over an al- 
phabet of size k, we can compute in time 2°( fc lo e( kn )) a 
finite union of linear sets with at most k generators and total 
size 2°( fc lo s™) (even when unary representations of num- 
bers are imposed). To this end, we establish a normal form 
theorem for semilinear sets: any semilinear set in N fc of size 
n (under unary representation of numbers) could be con- 
verted into an equivalent union of 2°( fclog ( fcn )) linear sets 
{vo + E^ =1 aiVi : £ N}, where numbers in Vo (resp. 
{v l } l =i) cannot exceed 2°( fclo s( fc "» (resp. 0(n)). Further- 
more, we show that this conversion can be performed in 
time 2°' fe lo s( kn )) . i n fact, a similar result is shown to hold 
for semilinear sets over Z (i.e. where offsets and generators 
are in Z fc ). The proof of the normal form theorem makes 
use of the well-known Caratheodory's theorem [33 1 from 
convex geometry. Our normal form theorem for semilin- 
ear sets are of independent interests and have applications 
beyond the computation of Parikh images of NFAs. 

To complement our upper bounds, we show that for ev- 
ery fixed k > 1 there exist infinitely many NFAs {A n } 
over an alphabet of size k such that A n has n states and 
its Parikh image contains 0,{n l ) linear sets. This implies 
that we cannot remove k from the exponent of the running 
time of translations from NFA to semilinear sets. Further- 
more, we give infinitely many CFG over a unary alphabet 
whose Parikh images contain at least 2" linear sets, where 
n is roughly the size of the CFG, and thus strenghthening 
the initial lower bound from 11241 . 

We give a few applications of our main results: (1) a 
new polynomial fragment of integer programming, (2) pre- 
cise complexity of membership for Parikh images of NFAs, 



(3) an answer to an open question posed by Abe [ 1 1 about 
polynomial PAC-learnability of semilinear sets, and (4) an 
optimal algorithm for LTL model checking over discrete- 
timed reversal-bounded counter systems, 

Finally, since there is a poly-time translation from reg- 
ular expressions to NFAs (e.g. see ll27l ). our upper bound 
result for NFAs transfer directly to regular expressions. Al- 
though NFAs are more succinct than regular expressions 
ifTTl and deterministic automata, our lower bound result also 
transfers to regular expressions and deterministic automata. 
Furthermore, since CFGs and pushdown automata are poly- 
time equivalent (e.g. see |27|), our result for CFGs directly 
transfer to pushdown automata. 

Related work It is of course well-known that semilinear 
sets also coincide with subsets of N fc expressible in Pres- 
burger arithmetic lfT2l (i.e., first-order logic over (N, +)). 
Such an alternative representation has indeed been fruit- 
fully exploited. In fact, using a technique developed by 
Esparza 11101 . Verma et al. Il32l has given a linear trans- 
lation from context-free grammars (equivalently, pushdown 
automata) to the NP-complete existential fragment of Pres- 
burger arithmetic expressing their Parikh images. Such a 
translation has been used to derive optimal algorithms for 
equational horn clauses l32l . This linear translation to exis- 
tential Presburger formulas is orthogonal to our result. On 
the one hand, our algorithm works only on NFAs and has 
exponential time complexity when the alphabet size is not 
fixed and therefore does not immediately yield an NP algo- 
rithm. On the other hand, existential Presburger formulas 
are exponentially more succinct than the classical represen- 
tation of semilinear sets (as a union of linear sets) and it is 
easy to check that the translation in ||32l does not produce 
output formulas in any known polynomial-time fragment 
of existential Presburger arithmetic even when the input is 
an NFA (or its equivalent CFG) over some fixed alphabet 
size. We will simply remark that many scenarios (includ- 
ing H3] HU [TU |2U] |26] [30l and the ones we consider in 
this paper) require the use of the classical representation of 
semilinear sets to derive optimal algorithms. 

Vector generalizations of the Frobenius problems (e.g. 
see ||25l and references therein) have also been proposed. 
Such generalizations are orthogonal to our results for semi- 
linear sets. For one, they mainly attempt to extend the no- 
tions of "conductors" from the original Frobenius problem, 
but neither attempt to reduce the number of generators nor 
obtain an efficient algorithm for doing so. 

Organization We fix our notations in Section [2] We 
prove a normal form theorem for semilinear sets in Section 
[3] which is applied in Section [4] to prove a general version 
of Chrobak-Martinez Theorem. Lower bounds are given in 
Section|5] and applications are given in Section|6] 
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2 Preliminary 

General notations Let N denote the set of nonnegative 
integers. We assume familiarity with basic notions from 
linear algebra: vector space, basis, linear independence, 
rank, etc. In the sequel, we use only the standard real vec- 
tor space HL fc . We use to denote the vector of M. k with 
all-zero entries. We denote by {ei} k =1 the standard ba- 
sis for M fc , where denotes the vector with all-zero en- 
tries except for the ith. We shall also require one nota- 
tion from the theory of convex sets 11331 . Given a finite set 
V — {vi, . . . , v m } of vectors in R k , let cone(V) be the set 
{E^iAiVi : Xi G K>o}. 

Partial orders Recall that a partial order ^ on a set S 
is well-founded if there does not exist a strictly decreasing 
infinite sequence s\ >- S2 >~ . ■ . of elements from S. An 
element s of S is said to be ^-minimal, if all s' G S with 
s' -< s satisfies s — s'. 

In the sequel, we shall reserve -< for the component- wise 
partial order on N k , i.e., (a%, . . . , Ofe) d: (bx, . . . , bk) iff 
a, < bi for all i G {1, . . . , k}. Dickson's lemma [8 | states 
that -< is well-founded. 

Automata An alphabet E is simply a finite set of letters. 
An NFA A is a tuple (E, Q, 8, qo, qp), where Q is a finite 
set of states, qo G Q is an initial state, qp G Q is the fi- 
nal state, and iJCQxSxQisa transition relation. A 
path tt in A from g G Q to q 1 G Q is simply an alter- 
nating sequence Poj3\Px . . . f3 m p m G (QE)*Q of states and 
letters such that po = q and p m = g'. In this case, we 
write L(7r) to denote the path labels f3\ . . . (3 m G £*, and 
say that 7r is a path on /3\ . . . /3 m . For convenience, we shall 
sometimes omit the path labels from tt, and simply refer to 
it as a path tt = po . . .p m on word u> = /3i . . . /3 m . In 
this case, the length \tt\ (resp. \w\) of the path 7r (resp. 
word w) is m. Given i, j G {0, . . . , 7r|} with i < j, we 
also write n[i, j] as the path segment piPi + \ ■ ■ - Pj of tt. 
Given two paths tt = p Px . . . p m and n' = p m f3 m+1 ...p n 
(with m < n), we let 7r tt' denote the concatenated path 
PoPi ■ ■ -PmPm+i ■ ■ - Pn- The NFA A is said to accept the 
word w G E* if there exists a path tt in A from qo to qp 
such that L(tt) = w. In this case, 7r is said to be an ac- 
cepting path. The language L(A) of A is the set of words 
accepted by A. 

Given a word w G S* and a G S, we write |iy| a to 
denote the number of occurences of a in w. In the sequel, 



we tacitly assume that there is a linear ordering < on S, 
i.e., E = {a±, . . . , ctk} with ctfj < <x, iff i < j. In this case, 
given a word tu G E*, we define the Parilch image V(w) 
of u; to be the tuple (|iy| ai , . . . , |w| afc ) G N fe . In addition, 
given a set L C E*, we define the Parikh image V(L) of L 
to be the set {P(w) : w G L} C N fc . We shall also write 
V(tt) for a path 7r in an NFA to denote V(L(tt)). 

Matrix notations Given two n-by- n 0-1 matrices M = 
K,j]nxn an d = [m-^nxn, we write M • M' to de- 
note the matrix M" = [m " 3 ■] „ x „ with m = Vfc=i( m i,fcA 
.). The operator • is often referred to as boolean ma- 
trix multiplication, which can easily be evaluated in 0(n 3 ). 
We also write M V M' to denote the application of the 
boolean operation V component-wise, i.e., resulting in a 

matrix M" = [m" ,-] n xn with m 'lj = m ij v m 'i,j- ^ n me 
sequel, we shall also write M[i,j] for the (t, j)-component 
niij of M. 

Semilinear sets For every vector v G Z fe and every finite 
set S = {ui, . . . , u m } of vectors in Z fe , we write P(v; S) to 
denote the "L-linear set {v + E^aiUi : ai, . . . , a m G N}. 
The pair B := (v; S) is said to be a linear basis for 
P(\;S). Notice that there exist non-unique linear bases 
for a Z-linear set. The vector v is said to be the offset of 
B, and the vectors S the generators of B. For v = 0, 
we also use cone^S) to denote P(v; 5). A Z-semilinear 
set S is simply a finite (possibly empty) union of Z-linear 
sets P(\i; Si), . . . , P(\ s ; S s ). In this case, we say that 
B = {(vj; Si)}| =1 is a semilinear basis for P(6) := 5. 
Likewise, semilinear bases for S are not unique. A Z- 
semilinear set S C Z fc is said to be "H-semilinear (or simply 
semilinear) if it has a semilinear basis with vectors from N fc 
only. The notion of N-linear (or simply linear) sets is also 
defined similarly. 

Since Z-(semi)linear bases £> are simply a sequence of 
vectors from Z fc , we could talk about their size ||£>|| when 
represented on the tapes of Turing machines. We shall use 
both unary and binary representations of numbers, and be 
explicit about this when necessary. In the sequel, we shall 
not distinguish (semi)linear sets and their bases, when it is 
clear from the context. Thus, we shall use such a phrase 
as "compute a (semi)linear set" to mean that we compute a 
particular (semi)linear basis for it. 

Arithmetic on 2 zfc Given two sets S\,S2 C Z fe , we de- 
fine an operation '+' on them as follows: Si + S2 '■= 
{vi + V2 : Vi G Si,V2 G S2}. Suppose now that 
Si = \JUi p (y* V i) and ^2 = U' : l'<»r\\\:. Then, 
observe that 51+5*2= U[=i U*=i ^(^ + u Wf). 

For si G 5*i, we shall also write s\ + S2 to mean {si} + S^- 
Thus, we have P(v; S) = v + cone^S 1 ). 
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3 Normal form for semilinear sets 

In this section, we shall prove a normal form theo- 
rem for semilinear sets: given a semilinear basis B in 
N k represented in unary, we can compute in time polyno- 
mial in and exponential in k another semilinear basis 
B' = {{y/ l \S l )} n l l =l in unary such that P(B) = P(B') 
and | 1 < k for each i € {1, ...,m}. For simplic- 
ity, we shall state only a general theorem for Z-linear sets 
of the form conewlV), where V is a finite subset of Z fc ; 
this can easily be used to derive the desired normal form 
theorem for semilinear sets (among others). [Recall that 
P{\;V) = v + cone N (y).] 

Theorem 3.1 Let V := {vi,...,v m } C Z fe \ {0} with 
m > 0. Let a G N be the maximum absolute value of 
numbers appearing in vectors of V. Then, it is possible 
to compute in time 

2 0(k\o g (m)+k 2 log(fco)) a sequence fZ- 
linear bases (wi; Si), ... , (w s ; S s ) such that 

s 

com N (V) = \JP(w i ;S i ) 

i=l 

where the maximum absolute value of entries of each Wj is 
0(m(k 2 a) 2k+3 ), each Si is a subset ofV with \Si\ < k, 
and s = 0(m 2k (k 2 a) 2k2+3k ). Furthermore, if V C N fe , 
we have {wi, . . . , w s } C N fc . 

Observe that this theorem causes only an exponential blow- 
up in the dimension k. Moreover, each set Si con- 
tains at most k generators. To prove this theorem, we 
start with a slight strengthening of the conical version of 
Ca rathe odory's theorem from the theory of convex sets 11331 
Proposition 1.15]. Proof is given in the appendix. 

Lemma 3.2 Let V := {vi, . . . , v m } C Z k \ {0} with m > 
0. Let a £ N be the maximum absolute value of numbers 
appearing in vectors of V. Then, it is possible to compute 
in time 2°( fcl °g™+ lo s a ), a sequence Si,...,S r of distinct 
linearly independent d-subsets ofV, where d G {1, . . . , k} 
is the rank of V, and 

r 

cone(V) = [J cone(Sj). 

i=l 

Let us first explain the idea behind the rest of the proof of 
Theorem l3.ll Intuitively, Lemma [3~l2l says that cone(V) C 
M. k can be subdivided into smaller subcones with exactly 
d G {1, . . . , k} generators where d is the rank of V. This 
lemma immediately gives an upper bound for coneN(V) as 
the union of the integer points in cone(Si); in general, the 
latter contains much more points than conej^V^). On the 
other hand, we have U£ =1 coneys';) C conepjiV), where 



the inclusion is strict in general. It turns out that an equality 
can be achieved by first making a "few" duplicates of each 
conej^Si) and then shifting them appropriately by some 
"small" integer vectors. 

We now prove Theorem 13. II First invoke Lemma [3721 on 
V and obtain linearly independent d-subsets Si, . . . , S r of 
V, where d = rank(V) and r < m k , satisfying cone(^) = 
ULi «me(5j). Then, it follows that cone(^) n 1 k = 
Uj=i (cone(5j) fl Z fc ). To compute the integer vector 
"shifts", we shall need to define the notions of canonical 
and minimal vectors. 

Characterization via canonical and minimal vectors 

Suppose now that v G cone(5j) n Z fc and Sj = 
{ui, . . . , Ud}- We make several simple observations: 

(01) There exists a unique vector [v] G {— ka, . . . , ka} k n 
cone(Sj) and unique non-negative integers aj, . . . , ad 
such that: 1) v = [v] + Sf =1 aiUi, and 2) [v] = 
Y,f =1 biUi for some (unique) < b\, . . . , bd < 1. To 
see this, observe that by linear independence of Sj 
there exist some unique Ai, . . . , G M>o such that 
v = E^jAjUj. Simply let a, := L^iJ, b t := \ { - a i; 
and [v] := £f =1 &iUj. Uniqueness is immediate from 
uniqueness of Ai, . . . , A^. 

(02) Given v' G cone(S,) n Z fc , we write v ~ v' iff 
[v] = [v']. It is easy to see that ~ is an equivalence 
relation of finite index (there are at most (2ka + l) k 
equivalence classes). If [v] = v, the vector v is said to 
be a canonical representative of the equivalence class 
{u G cone(5j) n Z fe : [u] = v}. In this case, we will 
also call v an Sj -canonical vector, or simply canonical 
vector when Sj is understood. 

(03) If v is in coneN(U) n cone(Sj), then v + £^ =1 CjUj G 
conepf(V) n cone(Sj) for every ci,...,qgN. 

We shall now use these observations to define a natural 
well-founded partial order <j on conejj^) n cone(Sj); 
note that coneN(V) n cone(Sj) =/= 0. Given v,w G 
coneN(^) H cone(S'j), we write v <j w iff, for some 
(unique) Sj-canonical vector vo and some (unique) coef- 
ficients ai, . . . , ad G N and bi, . . . , bd G N, it is the case 
that: 1) v = Vo + Sf =1 aiUj, 2) w = v + £^ =1 &iUj, and 3) 
(oi, . . . , ad) d {bi, ■ ■ ■ , bd)- The following simple lemma 
(proof in the appendix) shows that <j is a well-founded par- 
tial order, and characterizes <j-minimal elements. 

Lemma 3.3 The relation <j is a well-founded partial or- 
der on coneN(U) fl cone(5j). Furthermore, a vector v G 
coneN(V^) n cone(iSj) is <*j-minimal iff none of the vectors 
(v - Ui), .... (v — Ud) are in coneN(V^) n cone(jSy). 
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Lemma l3~3l and Observation (03) immediately implies that 
coneN(V) is a union of linear sets P(v; Sj) taken over all 
j = 1, . . . , r and <j -minimal vectors v. 

Lemma 3.4 conepjtV) = UJ=i Uv ^( v ' &j)> where v is 
taken over all <j -minimal vectors. 

Proof. Q) Obvious. 

(C) If v G coneN(V), then v G cone(Sj) n Z fc for some 
j G {1, . . . , r}. By Lemma [331 there exists a <j -minimal 
vector v' satisfying v' <j v. Observation (03) implies that 
veP(v';Sj). " □ 

Note also that if V C N fc , then all <^ -minimal vectors (1 < 
j < r) are also nonnegative. 

A roadmap for rest of the proof is as follows. We shall 
show that each <j -minimal vectors cannot be too large and 
can be efficiently enumerated. This will immediately give 
us the desired sequence of linear bases. The proof of this 
will require connections to integer programming, and the 
use of dynamic programming. 

Bounds via integer programming 

For each Sj — {ui, . . . , u^}, we shall now show that 
all <jj-minimal vectors v cannot be too large. To this 
end, for each canonical vector vo G cone(Sj) n con- 
sider the integer linear program Ax = \l (x t 0), 
where A is the k x (m + d) matrix consisting of columns 
Vi, V2, ■ ■ ■ , v m , — Ui, U2, .... Ud (in this order) and x 
is the column (to + d)-vector consisting of the variables 
x%, . . . , x m , yi, . . . , yd (in this order). The following sim- 
ple lemma (whose proof is in the appendix) shows that 
r< -minimal solutions to such integer programs — as we 
shall see, they cannot be too large as well — provide up- 
per bounds for how large <j -minimal vectors can be. 

Lemma 3.5 For every <j-minimal vector v G conepj(V) D 
cone(Sj), let \ := [v] and b = (&i, . . . , bd) G N d be the 
unique coefficients such that v = Vo + Ti k =l biVLi. Suppose 
also that c = (ci, . . . , c m ) is a ^-minimal solution to the 
integer program S™ x XiSi = v fx y 0). Then, the vector 
w := (c,b) G N rn+d is a ^-minimal solution to the integer 
program Ax = vo (x >z 0). 

Consider the set U of all vectors Vo + £f =1 a;Ui, 
where Vo ranges over all canonical vectors and a\, . . . , ad 
ranges over all d-tuples of nonnegative integers such that 
(ci, . . . , Cm, oi, . . . , ad) is a ^-minimal solution to the in- 
teger program Ax = Vo, for some ci , . . . , c m G N. We shall 
see now that the maximum absolute value B of numbers ap- 
pearing in U exist, which immediately gives an upper bound 
for the maximum absolute value of entries of <j ; -minimal 
vectors. The following general lemma, whose proof is a 



straightforward adaptation of the proof of ||2T1 Theorem p. 
767], yields an upper bound for B. 

Lemma 3.6 Let A be a k x n integer matrix and b a k- 
vector, both with entries in [— t, t] D Z, where t G N. Then, 
every ^-minimal solution x G N ra to Ax — b (x >z 0) is in 
{0,l,...,n(kt) 2k+1 } n . 

Notice that the maximum absolute value of numbers appear- 
ing in our integer programs cannot exceed t :— ak (which 
could appear on the right hand side of the equation). If 
M := (to + k)(kt) 2k+1 , it follows that B < akM + ak < 
N := (m + k)(k 2 a) 2k+2 + ak. This completes the proof of 
existence for Theorem 13. H and gives us the desired bounds 
for the parameter s and the maximum absolute value of en- 
tries of each w, in Theorem 13. II It remains to show how to 
make this algorithmic. 

Computing canonical and minimal vectors 

We first show how to compute all the canonical vectors. 
Since Gaussian-elimination over rational numbers can be 
implemented to run in time polynomial in the total number 
of bits in the input matrix [9| and that each Sj is linearly 
independent, we could easily compute all 5j -canonical vec- 
tors (for all j G {1, . . . , r}) in time 2°( felo g( feQ )+ fcl °s m ) by 
going through all candidate vectors v G {— ka, . . . , ka} k 
and checking whether there exist < b\,...,bd < 1 such 
that £^ =1 &iUi = v. [Transform into row-reduced eche- 
lon form to compute the unique solution, if exists. Since 
Sj U {v} C Z fc , the coefficients b%, . . . , bd will be rational.] 
For each fixed j G {l,...,r} and each fixed Sj- 
canonical vector Vo, we now show how to compute the set 
of all <j -minimal vectors v such that [v] = Vo by dynamic 
programming in time 2°( felo s m + fc2 lo s( fca )). Observe that 
since there are at most r(2ak + l) k = 2 0( - klo s( kam )) possi- 
ble Vo, doing this for all canonical vectors would take time 
2 o(fcio g m+fc 2 lo g (fca)) ; which is also the total com piexity of 

the algorithm. To this end, we first fill out in stages a table 
T\ which keeps track of all vectors v G {0,1,...,./V} fc n 
coneN(V^). At stage h = 1, 2, . . . , m, we collect all vectors 
v that can be written as E^ =1 CiVj, where < c; < M. Since 
the size of the table is at most N k (k log N) — fclogA^ 
bits are used to identify each element in the table with 
an associated fc-tuple — this could be carried out in time 
0(m(N k (k\ogN)) 2 ) = 2°( fcl °g m + fc2l °g( fca )). We then 
fill out in stages another table T<z, which keeps track of all 
vectors v G {0, 1, . . . , N} k nP(\ a ; Sj). This could be done 
in d stages, similar to the computation of T\, and could 
be implemented to run in time 0(k(N k (k log N)) 2 ) = 
2 o(k\o gm +k 2 log(fea)) We then simp i y compute a new table 
Ts = T\ fl Ti, from which we eliminate vectors that are not 
<j ; -minimal by using the characterization of <j -minimal 
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vectors from Lemma 13.31 All in all, this could be imple- 
mented to run in time 2°( fcl °s m + fc2 lo s( fcQ )). 

4 Parikh Images of NFAs 

In this section, we shall apply Theorem l3.1l to obtain the 
main result for Parikh images of NFAs. 

Theorem 4.1 Let A be an NFA with n states over an al- 
phabet £ of size k. Then, there exists a representation of 
V{A) as a union of linear sets P(vi; Si), . . . , P(v m ; S m ), 
where the maximum entry of each Vj is 0(n 3fc+5 fc 4fe+6 ), 
each Si is a subset of {0, . . . , n} k with \Si\ < k, and 
m = 0(n k + 3fe + 5 fc 4fc + 6 ). Furthermore, this is computable 
in time 2°( fc2 M*™)). 

Observe that this theorem causes an exponential blow-up 
only in the size of the alphabet. Efficiency could be im- 
proved by outputting numbers in binary. 

We shall devote the rest of this section to prove this the- 
orem. Let A = (£, Q, 5, qo, q F ) be a given NFA, where 
\Q\ = n and £ = {ai, . . . , o^-}. Throughout the proof, we 
shall use the notion of "cycle type". A cycle type is a Parikh 
image v £ N k of any word w £ £-" such that there is a 
path 7r of A on w from some (not necessarily initial) state p 
to itself. The cycle tt is said to witness v. Observe that the 
sum of the components of any cycle type cannot exceed n. 

Characterization of V{L(A)) 

We shall first give a characterization of the Parikh image of 
A in terms of Parikh images of "short" paths together with 
some cycle types. Given a path tt = poPiPi ■ ■ ■ f3 r Pr of A 
from the (not necessarily initial) state p n to the state p r , let 
S n =C {0, . . . , n} fc be the set of all the cycle types that are 
witnessed by some cycles C — p' p'i ■ ■ ■ p' t po in A such that 
Pi = Pj f° r some * £ {0, • ■ • , t} and j £ {0, . . . , r}. That 
is, C and ir meet at state p' i = pj. Now define T n to be the 
linear set P(P(tt); S n ). 

Lemma 4.2 The following identity holds: 

V(L(A)) = \JT„, 

where tt is taken over all accepting path of A of length at 
most (n — l) 2 . 

Proof. (C) Assume that v £ V{L(A)) and a = 
PoftiPi ■ ■ ■ PrPr be an accepting path in A such that V{a) = 
v. We shall construct another accepting path a 1 in A of 
length at most (n — l) 2 . For each state q occuring in a, let 
l(q) be the last (i.e. maximum) index i £ {0, . . . , r} such 
that pi = q. Let us write down all such l{q) in an increasing 



order, e.g., i$ < ii < . . . < i s = r. Note that s < n. By a 
standard result in graph theory, each subpath a\ij, ij+i] of 
a can be decomposed into a simple path ttj from p ij to Pi j+1 
of length at most n—1 and finitely many simple cycles (pos- 
sibly with duplicates) C\ , . . . , C/,. each of length at most n. 
That is, we have V{ir[ij,ij +1 ]) = V(iCj) + Y, 1 } =1 V{C l ). 
Such a decomposition result, however, might allow some 
cycle Ci to avoid (i.e. not meet with) ttj as can be wit- 
nessed from ||29l Figure 2]. On the other hand, Ci must 
visit some states of pi , . . . , pi s as these contain all states 
in a. Thus, we simply define a' to be the accepting path 
7To 7Ti . . . 7r s _i of length at most (n — l) 2 . It follows 
that v £ T CT '. 

Q) Conversely, let v £ for some accepting path ir in 
A of length at most (n — l) 2 . Then, if S n = {vi, . . . , \ s }, 
thenv = V{-K) + T, s i=1 ai\i forsomeai, . . . ,a s £ N. LetC^ 
be a cycle in A that meets with tt and satisfies V(Ci) = \i. 
We can construct an accepting path a in A with V(a) = v 
as follows: start from tt as the "base" path, and for each 
i £ {1, . . . , s}, attach ai copies of C,; to one pre-selected 
common state of C,; and tt. □ 

As an immediate corollary, we have: 

Proposition 4.3 Let A be an NFA with n states over an 
alphabet £ of size k. Then, V(A) can be represented as 
a union of linear sets P(vi; Si), ... , P(v m ; S m ), where 
\i £ {0, . . . ,{n — l) 2 } fc and the components of each vector 
in Si cannot exceed n. 

Remark: A slightly stronger version of this proposition was 
claimed in [26 1, where the maximum component of each \i 
cannot exceed n. Their proof turns out to have a subtle error 
that also occurs in the proof of Chrobak-Martinez Theorem 
BED], which was recently fixed in [29 1 . In fact, we show in 
Proposition |5.3| that our quadratic bound is essentially opti- 
mal, i.e., it cannot be lowered to o(n 2 ). (End Remark) 

Observe now that the proof of existence in Theorem |4.1| 
is essentially immediate from Proposition 14.31 and Theo- 
rem 13.11 In fact, if we are only interested in existence, 
this yields a better upper bound of 0(n 3 ( fc+1 )fc 4fc+6 ) (resp. 
0(n k + 3fe + 3 k 4k+6 )) for the maximum entry of all offsets 
{ v iK™ i (resp. to) in the statement of Theorem l4. 1 1 

Dynamic programming algorithm 

It remains to show that this can be made algorithmic. To this 
end, we first show how to compute all the cycle types of A. 
More precisely, let Q = {q , . . . ,q n -i}, where q n -i := 
q F , and let / = {(h,...,b k ) £ N fc : EfL^ < n}. For 
each vector v £ N fe , we write M v = [aij] nxn for the n-by- 
n 0-1 matrix where a,j = 1 iff there exists a path tt from 
qi to qj with V{tt) = v. We are interested in computing 
all matrices M v for each v £ /. Observe that the naive 
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algorithm, which runs through all paths of A from qi to qj 
with V(tt) — v, has time complexity that is exponential in 
n. We will give an algorithm for computing these in time 
20(fciogn) usm g dynamic programming. To this end, let us 
derive a recursive formula for computing M y based on M x , 
with v' < v. As base cases, we first observe that Mo is 
the n-by-n identity matrix. Furthermore, each matrix M ei 
could be constructed easily from the transition relation S. 

Lemma 4.4 Let\ = (ri,r 2 , . . . ,n-i, n + l,n + i, . . . , r*) 
with each Ti G N. Then, the following identity holds: 

M y = \J M u • M ei • M w 

where u ranges over all vectors -< v whose ith entry is 0, 
and w is the vector v — e^ — u. 

The proof of this lemma can be found in the appendix. Intu- 
itively, this recurrence relation can be derived by observing 
that a path tt with V(tt) = v can be uniquely decomposed 
into three consecutive path segments tti, TT2, and tt^ with 
V(tti) = u, V{t^2) = e.;, and 7-* (^3) = w, for some pre- 
scribed u and w. The following lemma, whose proof is also 
in the appendix, is a simple application of Lemma |4~4l and 
dynamic programming. 

Lemma 4.5 We can compute {M v } ve / in time 2°( felo s "). 

For each i G {0, . . . , n — 1}, let be the set of all cycle 
types v witnessed by some cycle tt = popi . . . po in A with 
Pj = qi. Since {M v } v e7 have been computed, all sets Ti 
could be computed within 0(n k+1 ) extra time. 

We now show how to compute V(L(A)) in time 
20(fe log(fcn)) j Q tn j s elK j we sna jj use anotner applica- 
tion of dynamic programming based on Lemma 14.21 The- 
orem [XT] and the sets {T^Zq, which we already com- 
puted. For each < i < (n — l) 2 and each < j < n, 
let Tij := (J where tt is taken over all paths in A of 
length i from qo to qj . By Lemma 14.21 it is the case that 

V(L(A)) = U!"o T i,n-i; recall that q n ^ = q F by def- 
inition. We shall now derive a recurrence relation for Tij. 



Lemma 4.6 It is the case that Tq q — {0} and T Q j = 

for each j G {1, . . . , n — 1}. Whenever i > and j G 
{0, . . . , n — 1}, we have 

T ij= U \ T i-^+ U pm)) . 

h=0 \ l<l<k,(q h , ai , q] )£S I 



This recurrence relation can be derived by observing that 
every path tt of length i from go to qj can be decomposed 
into the path tt[0, i — 1] ending at some state qh and the path 
Tr[i — 1, i] = qhaiqj. The cycle types Tj can be "used" since 
qj is visited. The proof is in the appendix. 

To finish the proof of Theorem 14.11 it suffices to give 
an algorithm with running time 2°( fc lo s( fcn )) for comput- 
ing a desired semilinear basis Pij for each set T^ . The 
algorithm runs in (n — l) 2 + 1 stages, where at stage j — 
0, . . . , (n — l) 2 the set Pij is computed. Obviously, we first 
set P ,o = {0} and P j = for each j G {1, . . . , n - 1}. 
Inductively, suppose that Pi h = {(v' 1 ; Sg)} 1 £^ 1 has been 
computed for each h E {0, . . . ,n— 1}. We will show how to 
compute Pi+i.j for any given j G {0, . . . , n — 1}. For each 
/i G {0, . . . , n — 1}, let Jh denote the set of numbers I G 
{1, . . . , k} such that (q^, ai,qf) G <5. Therefore, we have 

Pi,h+{J leJh PieuTj) = Ur=iU eJh P^+e^ur,). 
We use the algorithm from Theorem 13.11 to compute an- 
other semilinear basis for each P(\g + e;,5^ U Tj) and 
then compute unions in the obvious way to obtain Pi+ij 
(note: duplicate is removed). The output of this algorithm 

is P = (Ji=i 1 ' ) Pi.n-i- The correctness of the algorithm is 
immediate from Lemma l4~6l 

We now analyze the time complexity of this algorithm. 
By induction, it is easy to see that at every stage of the 
algorithm S' s l U Tj C {0, . . . ,n} k holds for each h G 
{0, . . . , n — 1} and s G {1, . . . , m^}. Therefore, the max- 
imum component over all offsets in the semilinear basis 
Pi+i.j is at most a + 0(n 3 ( fe+1 )fc 4i;+6 ), where a is the max- 
imum entry in each + e; over all h G {0, . . . , n — 1}, 
I G Jh, and s G {l,...,m^}. Note that the sum- 
mand 0(n 3 ( fc+1 )fc 4fc+6 ) is due to an application of Theo- 
rem 13.11 By induction, at stage i the maximum compo- 
nent over all offsets in {Pi,j}jZo is i x 0(n 3( - k+1 '>k ik + 6 ). 
This means that the maximum entry of each offset in P 
is 0(n 3fc+5 fc 4fe+6 ), and the number of linear bases in P is 
0(n k + 3fc + 5 fc 4fe + 6 ) (since duplicates are always removed). 
It is also easy to see that at each stage i, the algorithm 
runs in time 2°( fc lo g( nfe )) 5 primarily spent in the algorithm 
from Theorem 13.11 All in all, our algorithm runs in time 
2°( k lo g( nfc )) ? which is also the complexity of the entire 
procedure. 

5 Lower bounds 

In this section, we shall prove three lower bounds to 
complement the results in the previous section. We start 
by proving that every semilinear basis for the Parikh image 
of a deterministic finite automaton (DFA) can be large in 
the size of the alphabet. 

Proposition 5.1 For each k G Z>o and each integer 
n > 1, there exists a DFA A n _k over the alphabet := 
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{ai , . . . , a&} with n + 1 states whose Parikh image contains 
at least n k ~ 1 /(k — 1)1 linear sets. 

Proof. Let A n . k = (Q = {q , . . . , q n }, 6, q , q n ) with 
S(qi,a) = qi+i for each a G and < i < n. This 
automaton has a finite language L(A n ,k) with Parikh image 
V(L{A n &)) containing precisely all ordered integer parti- 
tions of n into k parts. Since the set V(L(A n &)) is finite, 
each ordered integer partition (n l7 . . . , n&) of n must ap- 
pear in precisely one linear set. Finally, it is easy to check 
(e.g. see OTl Chapter 13]) that the number of ordered par- 
titions of n into k parts equals (""j^ 1 ) > n k ^ 1 /(k — 1)1. 

a 

This proposition implies that, for every fixed k > 1, there 
exists infinitely many DFAs {A n } over an alphabet of size k 
where A n has size 0(n) but V(A n ) must contain f2(n fc_1 ) 
linear bases. Therefore, this shows that k cannot be re- 
moved from the exponent in Theorem 14. II In addition, ob- 
serving that the DFAs that we constructed have equivalent 
regular expressions of size 0(n), Proposition ^, ll also gives 
lower bounds for Parikh images of regular expressions. 

Next, we show that Theorem 14. II cannot be extended to 
CFGs (equivalently, pushdown automata). More precisely, 
we show that the number of linear sets for Parikh images of 
CFGs could be exponential in the size of the CFGs. 

Proposition 5.2 There exists a small constant c G Z>o 
such that, for each integer n > 1, there exists a CFG G n of 
size at most cn over the alphabet £ := {a} whose Parikh 
image contains precisely 2™ linear sets. 

Proof. We will construct a CFG G n such that 
V{L{G n )) = {0,1,..., 2" - 1}, each of whose el- 
ements will appear in precisely one linear set. Our 
construction uses the lower bound technique in 11241 . 

Our CFG G n contains nonterminals S, {Ai}"~Q, and 
{Bi}"~Q, and consists precisely of the following rules: 

S -)■ A ...A n ~i 
Ai — > e for each < i < n 
Ai — > Bi for each < i < n 
Bi — > Bi-iBi-i for each < i < n 
B Q -> a 

The initial nonterminal is declared to be S. It is easy to 
prove by induction that, for each word w G £*, Bi =>* w 
iff w = a 2 . This implies that Ai generates either e or a 2 . 
Thus, we see that L(G n ) = {a 1 : < i < 2™}, which 
easily yields the desired result. □ 

Finally, we give a lower bound proving the tightness of 
quadratic upper bound in Proposition 14.31 even when re- 
stricted to DFAs. 



Proposition 5.3 For each positive integer n > 2, there 
exists a DFA A n with 2n + 3 states such that if 
V(L(A n )) can be represented as a union of the linear sets 
P(vi; 5*i), . . . , P(v r ; S r ), then one entry in some \i is at 
least n(n + l)/2. 

This proof is given in the appendix. In fact, the constructed 
DFA A n has an equivalent regular expression of size 0(n) 
as well, therefore yielding the same quadratic lower bound 
for regular expressions. 

6 Applications 

Integer Programming 

Integer programming (IP) is the problem of checking 
whether a given integer program Ax = b (x >; 0), where 
A is an k-by-m integer matrix and b G Z fc , has a solution. 
It is well-known that this problem is NP-complete [22|. On 
the other hand, for k = 1, there is a pseudopolynomial- 
time algorithm for this problem (a.k.a. knapsack problem), 
which remains NP-complete under binary representation of 
input numbers [22 1 . Furthermore, Papadimitriou [21] has 
established a pseudopolynomial-time algorithm for solving 
IP, for any fixed k > 1. The running time of his algorithm 
is 2°( fclo s m+fc2lo s( fca )), where a is the maximum absolute 
value of numbers appearing in the input. Using our results 
in the previous sections, we could show that the problem 
remains poly-time solvable (for any fixed k > 1) even if the 
numbers in b are given in binary (and A in unary). 

Theorem 6.1 Given a k-by-m integer matrix A and a vec- 
tor b G Z fc , let a ( resp. b) be the maximum absolute value of 
numbers appearing in A (resp. b). Then, checking whether 
the integer program Ax = b (x >z 0) has a solution can be 
done in time 2°( felo s m + fc2 i°g(M+io g io g b)_ 

The proof of this theorem is in the appendix. It is essentially 
a simple application of Theorem 13.11 and Kannan's poly- 
time algorithm [ 17] for IP for a fixed number of variables, 
which improves the running time of Lenstra's original poly- 
time algorithm 1 19 1. Its running time on an integer program 
with d variables and input of length L is 0(d 9d L log L), 
which is polynomial even under binary encoding of num- 
bers in input. 

Theorem 16. H is known to be true when k = 1, by solu- 
tions to the Frobenius 's problem, and has such an applica- 
tion as the coin problem (e.g. see ll25l ). We give another 
not so obvious application of Theorem l6.1l in the appendix. 

Membership for Parikh images of NFAs 

The membership problem for Parikh images of NFAs is de- 
fined as follows: given an NFA A over £ — {a±, . . . ,a k } 
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and a tuple b £ N fe given in binary, decide whether b £ 
V(L(A)). A similar membership problem could be de- 
fined for CFGs (equivalently, pushdown automata). This 
problem is known to be NP-complete (e.g. see [10]). No- 
tice, however, that the hardness proof in ifTUll requires k to 
be unbounded. Furthermore, since it is well-known that 
NFAs can be efficiently converted to equivalent pushdown 
autommata (equivalently, CFGs), it follows that the mem- 
bership problem for Parikh images of NFAs is in NP. [See 
11261 [32l for an alternative proof via existential Presburger 
arithmetic] We first give two complementary lower bounds 
for these results. 

Proposition 6.2 Membership for Parikh image of NFAs 
(resp. CFGs) is NP-hard when k is not fixed (resp. even 
when k = 1). 

NP-hardness already holds for DFAs (reduction from the 
hamiltonian path problem) and for regular expressions (re- 
duction from one-in-three 3SAT). For CFGs over unary al- 
phabet, we use the encoding of large numbers given in 
Proposition l5.2l to encode the 0-1 knapsack problem. 

This leaves the complexity of the membership problem 
for Parikh images of NFAs for any fixed k > 1. Combining 
Kannan's poly-time algorithm for IP over a fixed number of 
variables (as in the previous application) and Theorem l4.ll 
it is easy to show that this problem is solvable in time poly- 
nomial in \A\ and ||b|| but exponential in k, i.e., polynomial 
when k is fixed. 

Theorem 6.3 Given an NFA A with n states over the al- 
phabet £ = {ax, . . . , afe} and a tuple b = (61, ... , b^) £ 
N fc written in binary with b := maxK^fcj^}, check- 
ing whether b G V(L(A)) can be done in time 

^0{k 2 log(fcn)+log log b) 

Polynomial PAC-learnability of semilinear sets 

Valiant's notion of PAC (Probably Approximately Correct) 
learning is a standard model in computational learning the- 
ory ([3l S) . In this framework, a learning algorithm is re- 
quired to run in time polynomial in the size of the training 
sample, and output a hypothesis for an (unknown) target 
concept that is as precise as the "user" desires, given any 
sufficiently large training sample (but still polynomial in the 
reciprocals of approximation/confidence parameters). 

The issues of PAC-learnability of semilinear sets have 
been addressed by Abe [1]. In particular, learning semi- 
linear sets of dimension 1 under binary representation of 
numbers is shown to be as hard as learning boolean formu- 
las in DNF, which is (still) a major open question in learn- 
ing theory. On the positive side, he shows that semilinear 
sets of dimension 1 and 2 can be poly-time PAC-learned 
when the numbers are represented in unary. To this end, 



he established a normal form lemma for semilinear sets of 
dimension 2 (in unary), which is simply a special case of 
Theorem 13. II for dimension 2. However, his proof makes 
use of geometric facts that are specific to R 2 . For this rea- 
son, he leaves open the learnability question of semilinear 
sets in unary over any fixed dimension k > 2 JT] Section 
9]. Replacing Abe's normal form lemma with Theorem l3.ll 
and following the proof of [ 1 Theorem 6.1], we can easily 
deduce the more general theorem. 

Theorem 6.4 Semilinear sets in unary representation over 
any fixed dimension k > 1 can be polynomially PAC- 
learned with respect to concept complexity. 

We shall not reproduce the proof for this theorem as it es- 
sentially requires only replacing Abe's normal form lemma 
with Theorem l3.1l in the proof of [fl] Theorem 6.1]. A short 
sketch is provided in the appendix. 

Verifying counter systems 

Minsky's counter machines are well-known to be a Turing- 
powerful model of computation. In verification litera- 
ture, many decidable subclasses of counter machines have 
been studied including reversal-bounded counter systems 
[15, 16 30], and their extensions with pushdown stacks and 
discrete clocks |7][30]. Intuitively, r-reversal fc-counter sys- 
tems are simply Minsky's counter machines with k coun- 
ters, each of which can change from an incrementing mode 
to a decrementing mode (or vice versa) only for a fixed r 
number of times. Their connection to our result is due to 
the use of Parikh's Theorem for obtaining decidability (ini- 
tially used in ifTBI ). Due to space restriction, we shall briefly 
mention only one corollary of our result. 

In [30 1, it was shown that model checking Linear Tem- 
poral Logic (LTL) over reversal-bounded counter systems 
with discrete clocks is solvable in double exponential 
time, even when one of the counters is free (not reversal- 
bounded). More precisely, if t is the number of clocks, n the 
number of states in the finite control, c the size (in binary) 
of the maximum number appearing in clock constraints, and 
\\tp\\ the size of the input LTL formula ip, then model check- 
ing LTL on such systems is decidable in time exponential 
in n but double exponential in c, k, r, \\<p\\, and t. This re- 
sult was derived by carefully analyzing the complexity of 
Ibarra's original result |[T6l of effective semilinearity of the 
Parikh images of languages recognized by reversal -bounded 
counter machines and replacing the application of Parikh's 
Theorem in [ 16 1 by the linear translation of [ 32 1 from CFGs 
to existential Presburger formulas expressing their Parikh 
images. By replacing the application of the translation of 
[32 ] by Theorem l4.1l and not allowing a single free counter, 
we can easily obtain the following upper bound. 
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Theorem 6.5 Model checking LTL over reversal-bounded 
counter systems with discrete clocks is solvable in time poly- 
nomial in n and exponential in c, k, r, ||y>||, and t. 

We shall not reproduce the proof (rather see the full version 
of the paper ll30l . which can be requested from the author). 
The time complexity given by this theorem is tight for the 
following reason: 1) LTL model checking over finite sys- 
tems is already PSPACE-complete |28|, 2) emptiness for r- 
reversal fc-counter automata is PSPACE-hard 1 15 1, 3) empti- 
ness for discrete timed automata is PSPACE-complete even 
when restricted to three clocks or when c is "small" (see Q 
and references therein). 
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APPENDIX 

A Proof of Lemma 13.21 

The conical version of Caratheodory's theorem ll33l 
Proposition 1.15] states that if we take V\,...,V S to be 
the sequence of all subsets of V with \Vi\ < rank(V) < 
k, then we have cone(U) = Ui=i cone(Vi). Let d := 
rank(V) and Si, . . . , S r be the set of all distinct linearly 
independent rf-subsets of V. It is not difficult to see that 
\J s i=1 cone(Vj) = U[=i cone(5j). In fact, we obviously 
have Ui=i cone(Vi) 2 Ui = iCOne(S'i). Conversely, ob- 
serve that for each linearly independent subset Vi of rank 
d! < d, one could add a vector v £ V \ Vi to Vi yield- 
ing a linearly independent (d 1 + l)-subset Vi U {v} of V; 
otherwise, each vector v £ V \ Vi is a linear combina- 
tion of vectors in Vi contradicting that d' < d. Repeat- 
ing this process several times, we obtain a linearly indepen- 
dent d-subset of V containing Vi. This easily implies that 
U- =1 cone(V^) C |Ji = i cone(5i). 

Observe now that r < (™) < m d < m k . So, we first 
compute the rank d of V by Gaussian-elimination in the 
standard way. It is known [9| that Gaussian-elimination 
over rational numbers can be implemented to run in time 
polynomial in the total number of bits in the input ma- 
trix. We then obtain Si,...,S r by going through each 
e?-subset of V and using Gaussian-elimination to check 
whether it has rank d. In total, the running time is 

<20(d log m+log(fcma)) ^^{k log m+log a) 

B Proof of Lemma 13.31 

That <j is a partial order is due to: 

• Observations (Ol) and (02): the uniqueness of choice 
of canonical vector Vo and coefficients a%, . . .,0,4 £ N 
for each vector v satisfying v = Vo + Sf =1 aiUi. 

• That ^< is a partial order on N k . 

To see that <j is well-founded, assume that there exists a 
strictly decreasing sequence vi t>j \2 >j ■ ■ ■■ Let v$ = Vo + 
E^ =1 a*Ui for some unique canonical vector Vo = [v<] and 
unique coefficients a 1 = (a\, . . . , aV) £ N d . In this way, 
we generate a strictly decreasing sequence a 1 >- a 2 >- . . . 
for the well-founded partial order >- on N k , and therefore a 
contradiction. Thus, < j is a well-founded partial order on 
coneN(V) n cone(S'j). 

Given a <j-minimal vector v £ conepjiV) n cone(Sj), 
it is obvious that none of the vectors (v — Ui), . . . , (v — u<j) 
cannot be in conejjtV) PI cone(Sj). Conversely, given a 
vectorv £ coneN(l^) ncone(S'j) which is not <j -minimal, 



we could find another vector v' £ coneN(^) D cone(5j) 
such that v' <j v. Using Observation (03), it is easy to 
show that at least one of the vectors (v — Ui), . . . , (v — U<j) 
is in coneN(V) D cone(Sj). 

C Proof of Lemma 13751 

That w is a solution is immediate. To 
show ^-minimality, consider a vector u = 
(c' 1 ,...,c' m ,b[,...,b' d ) £ N m+d such that u < w 
and Aa T = Vq. Define v' := Vo + Ef =1 6^Ui and thus 
v' = E™ ic'^i. This means that v' £ coneN(U) ncone(5j) 
and, by <j ^-minimality of v,it follows that v' = v and thus 

= hi for every 1 < i < d. That c is a ^-minimal solution 
to the integer program E^aiEjVj = V (v >z 0) implies that 

= Ct for every 1 < i < m and, thus, u = w. 

D Proof of Lemma |4l 

We only show that if the M v [j, h] = 1, then so is the 
(j, /i)-component of the matrix in the r.h.s. The converse 
can be proved by observing that all the steps below can be 
easily reversed. 

Let s = 1 + ^i =1 n. Suppose that tt = q^Piq^ . . . /3 s qi s 
is a path from qj to qh with V(n) = v. Thus, we have Iq = j 
and l s = h. We now decompose tt as follows. Let t be the 
first position where the letter on occurs in tt, i.e., f3 t = 
and P t > / Q4 for all t' < t. Let tt x := q^^ . . . q lt _ 1 , tt 2 := 
qit-iPtqit, and tt 3 := qi t (3 t+ i ■ ..qi s . Let u := V(tti) and 
w := V{tt^). Notice that the ith entry of u is and w = v 
e, u. Furthermore, we have M u [j, U-i] = M e . [h-i, h] = 
M w [lt, h] — 1. It follows that the (j, /i)-component of the 
matrix M„ • M ei • M w is 1 . 

E Proof of Lemma 14.51 

This can be done using Lemma 14.41 and dynamic pro- 
gramming. The algorithm has n + 1 stages. At stage 
j = 0, . . . , n, we compute all M y where the components 
in v sum up to j. These will be saved in the memory for 
subsequent stages of the iteration. As base cases, we would 
obtain Mo and M e ., for each 1 < i < k, directly from 
the input. Notice that boolean matrix multiplication can be 
done in 0(n 3 ) and so each stage of the computation can be 
performed in 0{n 2k+A ). Thus, the entire computation runs 
in time 2°( fclo s™). 

F Proof of Lemma HS1 

We first define the following notation: for every path tt in 
A, let denote the union of all Th over all h £ {0, . . . , n— 
1} such that qh occurs in tt. 
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Figure 1. A depiction of the DFA A 



(C) Let v G Ti j. Then, there exists a path tt = 
qi Q Pi ■ ■ ■ fiiqu such that v G T„, where Iq = and li = j. 
Let 7r' be the path segment n[0,i — 1] of tt. Thus, we 
have IV C IV Suppose that = {ui, . . . , u m } and 
IV = {ui, . . . ,u m '} for some m' < m. It follows that 
rV — r,,. , C Tj, In other words, new cycle type could 
only be introduced by visiting qj. Since v G T„, we have 
v = V(tt) + Y.'" l =1 a s u s for some oi, . . . , a m G N. Thus, if 
ai = ft, it follows that v = (P(tt') + ias u s ) + (ej + 
T,% m , +1 a s u s ). This establishes that v G T i>h + P(e h Tj) 
as desired. 

(D) This inclusion could be proved by observing that all 
the steps in the converse case could be easily reversed. 

G Proof of Proposition 15.31 

The automaton A n = (S n , Q n , S n , qo, qjr), where Q n = 

{Qo,Qf}U{po, •••>Pn}U{pi,...,K} andS n = {a,6,c}U 
{a,,e^ : 1 < « < n}. We specify the transition function 
5 n in Figure [T] Notice that A n has an equivalent regular 
expression R n of size 0(n). For example, when n = 2, we 
can define R n to be 

6(a(ai(a' 1 )*c|aa2(a2)*c))*6. 

We now argue that the a-component of some Vj 
must be at least n(n + l)/2. Let m be the maxi- 
mum entry over all vectors in Ui=i ^i- Define AT := 



(max-tl^l : 1 < i < r} " ( " 2 +1) TO^) + 1. For each 1 < i < 

n, let d be the cycle poapia . . . piaip'^a'^) 1 ^ cpo. Con- 
sider the accepting path tt = (qobpo)OCiQC2&- ■ -0C„0 
(pobqp). We have V(n) G P(vh' : Sh) for some 1 < h < r. 
Observe also that a occurs precisely = n(n + l)/2 

times in tt. 

Claim. Each a' r component of (1 < i < n) is nonzero. 

We now prove this claim. Let Sh — {ui, . . . ,u s } and 
V(tt) = \h + E* =1 ^Ui. For each i G {1, . . . ,n}, there 
exists a vector with a positive -component and tj i > 
n(n + l)/2; for, otherwise, the a'^ , -component of V{tt) is 

at most \Sh\ n ^ < N, a contradiction. In particular, 
this implies that all a-component of Uj it where a ^ a' { 
for all i G {1, . . . , n}, is as each such letter a occurs 
at most n(n + l)/2 times in tt. But this means that each 
ai-component of is nonzero; for, otherwise, we could 
consider the vector v;,, + Mj i which would not correspond to 
any accepting path in A n since at least one letter ai needs 
to read by A n if a\ is to occur in the path. This proves our 
claim. 

Now consider each word w = wq . . . wi G L(A n ) such 
that V(w) = v/j. It is easy to see that the number of oc- 
curences of a in w must be at least n(n + l)/2. In fact, for 
every 1 < i < n, define ji = minjj : Wj = ai}. For 
each i, the number of occurences of a in Wt ■ ■ ■ Wj i , where 
t = maxjjV : ji' < ji, 1 < i' < n}, is at least i. The lower 
bound of n(n + l)/2 on the number of occurences of a in 
w immediately follows. 

H Proof of Theorem gZD 

To prove this theorem, let \i (1 < i < m) be the 
i-th column vector of A and write x T = (xi, . . . ,x m ). 
Then, we have Ax — E™ -yXiXi and thus checking for 
solvability of Ax = b is equivalent to checking whether 
b G conewiV), where V = {vi,...,v m }. By Theorem 
E0 we obtain in time 2°( fcl °s m+fc2 lo s(ka)) some semi . 
linear sets P(Wi; S%), . . . , P(w r ; S r ) with coneN(V^) = 
Ui=i P( w ii Si), where the maximum absolute value of en- 
tries of each Wi is 0(m(k 2 a) 2k+3 ), each Si is a subset of 
V with \Si\ < k, and s = 0{m k+1 (k 2 a) 2k+z ). We would 
then simply check whether b G P(wi;Si) for some i = 
1, . . . , s. To this end, we employ Kannan's polynomial-time 
algorithm [17 | for IP for fixed number of variables, which 
improves the running time of Lenstra's original polynomial- 
time algorithm [ 1 9 1 . Its running time on an integer program 
with d variables and input of length L is 0(d 9d L log L). 
Observe now that each number in Si and w, could be rep- 
resented using 0(k log(mafc)) bits. Therefore, if Si = 
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{ui , . . . , Ud}, the integer program Y^f =1 XiUi = b w, has 
length L = log b + fc 3 log(mafc)) and uses d < k vari- 
ables. On this input, Kannan's algorithm will terminate in 
time 2 ( fclo s fc + lo s lo g b + lo s lo g( ma )). In the worst case, we 
will need to run Kannan's algorithm for each i — 1, . . . , s 
if the answer to the original IP question is negative. In total, 
the algorithm runs in time 2°( fc lo s m+fc2 i°s(M+i°g log b) _ 

I Another application of Theorem 16.11 

Another not immediately obvious application is the fol- 
lowing navigation problem. Suppose we have a robot in Z fe 
that can make finitely many different types of local moves 
Vi, . . . , y m 6 Z fc at any given point in Z fc . It is reasonable 
to assume that local moves are small (i.e. given in unary). 
We are interested in the following global behavior: when 
initially placed at 0, can it eventually arrive at a given vec- 
tor b? The vector b can be assumed to be large (i.e. given in 
binary) as it pertains to the result of a global behavior. The- 
orem |6T| implies that this navigation problem is poly-time 
solvable when the dimension k is fixed. 

J Proof of Proposition |6T2l 

DFA We now give a poly-time reduction from the hamil- 
tonian path problem to membership problem for Parikh im- 
ages of DFAs. The hamiltonian path problem asks whether 
a given graph G = (V = {v\, . . . , v n }, E) with a source 
s := vi G V and a target t := v n G V has a hamil- 
tonian path from s to t, i.e., a path from s to t in G that 
visits each vertices in V exactly once. Given G,s,t, we 
define the DFA Aa, s ,t — (S, Q, 6, qo, qF) where Q := 
V, £ := {cti, . . . ,a n }, qo '■= s, qF '•= t, and S := 
{(vi,ctj,Vj) : (vi,Vj) G E}. Then, it is easy to see that 
G has a hamiltonian path from s to f iff the Parikh im- 
age V{ol\ . . . a n ) of the word a± . . . a n is in V(Ac, s ,t) iff 
(0, 1, 1, . . . , 1) G V(Ac,s,t)- This completes the proof of 
NP-hardness of the membership problem for Parikh images 
of DFAs with unbounded alphabet size. 

Regular expressions One-in-three 3SAT is the follow- 
ing problem: given a boolean formula <p in 3-CNF, does 
there exist a satisfying assignment for ip that additionally 
makes no more than one literal true for each clause. We 
shall call such an satisfying assignment l-in-3. This prob- 
lem is NP-complete (see "M. R. Garey and D. S. Johnson. 
Computers and Intractability: A Guide to the Theory ofNP- 
completeness, W. H. Freeman, 1979."). We shall reduce this 
problem to the membership problem for Parikh images of 
regular expressions. Given ip = C\ A . . . A Ck, where Ci 
is a multiset over L := {x±,x\ . . . , x n ,x n } with \d\ = 3, 
we define a function / : L — } {1, . . . , k}* as follows: 



• f(xi) := a\...ak where a, := i if x, £ Cj, and 
a,i := e otherwise. 

• f(x~i) := a%...ak where en := % if xi G Cj, and 
a,i :— e otherwise. 

That is, the function / associates a literal with the indices 
of clauses that are satisfied when the value 1 is assigned 
to the literal. The corresponding regular expression over 
S = {1, . . . , fc} is 

R v = (f(x 1 )\f(x- 1 ))...(f(x n )\f(x- n )). 

Let 1 G We claim that tp is a positive instance of 

one-in-three 3SAT iff 1 G V(L(R V )), To prove this, sup- 
pose that tp is a positive instance with a l-in-3 satisfying 
assignment a : L — > {0, 1} (i.e. cr(xj) = 1 iff <t(x,) = 0). 
Consider the word w := X\ . . . X n G £*, where 

v- ._ / if O-(^i) = !. 

/(xl) ifa(x0 = 0. 

Observe that w G L(R ip ). Since cr is a l-in-3 satisfying 
assignment, it follows that V(w) = 1 and, therefore, we 
have 1 G P(L(R lp )). The converse direction can be proved 
by reversing the above construction of the word w. Finally, 
observe that the construction of R v and 1 can be done in 
time polynomial in the size of tp. 

CFG For CFGs over unary alphabet, we could easily use 
the encoding of large numbers given in Proposition |5.2| to 
succinctly encode the NP-complete 0-1 knapsack problem, 
i.e., given oi, . . . , a m G N and b G N all represented in bi- 
nary, decide whether there exists a subset S C {1, . . . , m} 
such that £i e sai = b. 

K Sketch for proving Theorem 16.41 

The reader should first familiarize themselves with the 
notions of PAC-learnability. Abe [1, Section 2-A] gives a 
highly-readable exposition; but a more gentle introduction 
could be found in ||3]|4). In the following, a semilinear set 
is said to have at most k generators if it is a union of linear 
sets with at most k generators. 

To prove Theorem 16.41 the reader should first recall 
the notion of polynomially generable natural union class 
(PGNU) in CD Definition 6.1-2]. [Definition 6.2 has a 
typo in item 2(ii): size(M") < size(M) should be 
size(M") < size(M').] By JTJ Lemma 6.1] and fl] 
Lemma 4.6], it suffices to show that for each fixed k > 1: 

(PI) semilinear sets in unary representation with at most k 
generators over any fixed dimension k > 1 is a PGNU 
(an adaptation of [ 1 Corollary 6.1]); and 
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(P2) semilinear sets in unary with at most k generators over 
N k is a Poly Blow-up Normal Form (c.f. [ 1 , Definition 
4.9]) for semilinear sets in unary over N fe (an adapta- 
tion of fl] Corollary 6.2]). 

The proof of (PI) is almost identical to the proof of U 
Corollary 6.1]: 

1. foranygivenfc G Z>i, there are at most k(b-\-l) k ( k+1 > 
semilinear sets over N k with at most k generators 
whose offsets and generators contain only numbers in 
{0, . . . , b}; and 

2. (First condition of PGNU) it is easy to see that check- 
ing whether a given tuple (ai, . . . , at) £ N k is a mem- 
ber of a given semilinear set with at most k genera- 
tors over N fc can be poly-time (truth-table) reduced to 
integer programming with fixed number of variables, 
which is poly-time solvable by Kannan's algorithm 
IfTTl (in this case, numbers could even be represented 
in binary). 

The next step is to show (P2). However, this is simply a 
corollary of our Theorem 13. II This completes the proof of 
Theorem l6.4l 
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